Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition

I Simple proof of linear convergence. I For convex functions, equivalent to several of the above conditions. I For non-convex functions, weakest assumption while still guaranteeing global minimizer. ? We generalize the PL condition to analyze proximal-gradient methods. ? We give simple new analyses in a variety of settings: I Least-squares and logistic regression. I Randomized coordinate descent. I Greedy coordinate descent and variants of boosting. I Stochastic gradient (diminishing or constant step-size). I Stochastic variance-reduced gradient (SVRG). I Proximal-gradient and LASSO. I Coordinate minimization with separable non-smooth term (bound constraints or L1-regularization). I Linear convergence rate of training SVMs with SDCA.